
United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 

Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-14S0 
www.uspto.gov 



APPLICATION NO. 


FILING DATE 


FIRST NAMED INVENTOR 


ATTORNEY DOCKET NO. 


CONFIRMATION NO. 


09/817,475 


03/26/2001 


Thiruvilwamalai Venkatraman Raman 


YOR92001000IUS1 


9819 



7590 10/23/2003 

Ryan, Mason & Lewis, LLP 

90 Forest Avenue 

Locust Valley, NY 11560 



EXAMINER 



TRAN, VINCENT V 



ART UNIT 



PAPER NUMBER 



2655 

DATE MAILED: 10/23/2003 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



Office Action Summary 



Application No. 

09/817.475 



Examiner 

Vincent v tran 



Applicant(s) 

RAMAN ET AL 



Art Unit 

2655 



The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )M Responsive to connmunication(s) filed on 26 March 2001 . 
2a)n This action is FINAL. 2bM This action is non-final. 

3) n Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1. 453 O.G. 213. 
Disposition of Claims 

4) n Claim(s) is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) n Claim(s) is/are allowed. 

6) 13 Claim(s) 1-25 is/are rejected. 
?)□ Claim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

AppHcation Papers 

9) 0 The specification is objected to by the Examiner 

10) 13 The drawing(s) filed on 26 March 2001 is/are: a)l3 accepted or b)^ objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

11) n The proposed drawing correction filed on is: a)^ approved b)^ disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) 0 The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) 0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)nAII b)n Some*c)n None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) n Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 . 

Attachment(s) 

1 ) 13 Notice of References Cited (PTO-892) 4) □ Interview Summary {PTO-41 3) Paper No(s). . 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) D Notice of Informal Patent Application (PTO-1 52) 

3) S Information Disclosure Statement(s) {PTO-1449) Paper No(s) 4. 6) □ Other: 



U.S. Patent and Trade marlt Office 
PTOL-326 (Rev. 04-01) 



Office Action Summary 



Part of Paper No. 5 




Application/Control Number: 09/817,475 Page 2 

Art Unit: 2655 

DETAILED ACTION 
Claim Rejections - 35 USC § 102 

1 . The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
states. 

2. Claims 1-5, 12-16, 21 and 25 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Ashby, III et al. (U.S. Patent No. 5,852,803). 

Referring to claim 1, Ashby, III et al. disclose a method of using speech for 
marking and subsequently identifying one or more items having electronically-readable 
identifiers respectively marked thereon, the method comprising the steps of: 

inputting at least a portion of the electronically-readable identifier marked on an 
item (barcode, col .2, In. 1-5; col.5, ln.31-33and col.9, ln.31-36); 

inputting from a user a spoken utterance that corresponds to the item 
(voice information, col.1, ln.63 -col.2, ln.1 and col.5, ln.33-35); 

associating the electronically-readable Identifier (bar-code label) input from the 
item with the spoken utterance input from the user (col.3, ln.36-39; col.5, In. 18-34 and 
col.9, ln.60-65); and 

outputting the spoken utterance when the electronically-readable identifier 
associated with the spoken utterance is subsequently inputted (col.3, ln.36-52 and col.9, 
ln.31-47). 



Application/Control Number: 09/817,475 Page 3 

Art Unit: 2655 

Referring to claim 2, Ashby, III et al. further disclose the method, wherein the 
electronically-readable identifier marked on the item is a universal product code (col .9, 
ln.31-34). 

Referring to claim 3, Ashby, III et al. further disclose the method of reading the 
universal product code from each item with a barcode reader (col.9, ln.32-36; col. 10, 
ln.21 -31 and Fig.1 1 , element #80). 

Referring to claim 4, Ashby, III et al. further disclose the method, wherein the 
associating step comprises storing the spoken utterance corresponding to each item 
using the universal product code read by the barcode reader for each Item as an index 
(memory, col.9, ln.31-47 and col. 11, In. 1-3). 

Referring to claim 5, Ashby, III et al. further disclose the method, wherein the 
outputting step comprises: 

subsequently reading the universal product code from an item with the barcode 
reader (col.9, ln.31-34); 

searching stored spoken utterances using the universal product code as an index 
(col. 10, ln.66 - col.11, ln.3) ; and 

playing back the spoken utterance that is found in the search to the user (col.1 1 , 
ln.3-7). 
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Referring to claim 12, Ashby, III et al. disclose a system for using speech for 
marking and subsequently identifying one or more items having electronically-readable 
identifiers respectively marked thereon, the system comprising: 

a first input device (bar-code reader, col.9, ln.32-36; col. 10, ln.21-31 and fig.11, 
element #80), the first input device being operative to input at least a portion of the 
electronically-readable identifier marked on an item (bar code, col.2. In. 1-5; col.5, ln.31- 
33 and col.9, ln.31-36); 

a second input device (microphone, col.6, In. 38 and fig.1 1, element #14), the 
second input device being operative to input a spoken utterance from a user that 
corresponds to the item (col.1, ln.63- col.2, ln.1 and col.5, ln.33-35); 

a storage mechanism (memory), the storage mechanism being operatively 
coupled to the first and second input devices and operative to associate the 
electronically-readable identifier input from the item with the spoken utterance input 
from the user (col.3, ln.36-39 and col.1 1 , In. 1-3); and 

an output device (speaker, fig.1 1 , element #36), the output device being 
operatively coupled to the storage mechanism and operative to output the spoken 
utterance when the electronically-readable identifier associated with the spoken 
utterance is subsequently inputted (col.3, ln.36-52; col.9, ln.31-47 and col.1 1, ln.3-7). 

Referring to claim 13, Ashby, III et al. further disclose the system, wherein the 
electronically-readable identifier marked on the one or more items is a universal product 
code (col.9, ln.31-34). 
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Referring to clainn 14, Ashby, III et al. further disclose the system, wherein the 
first input device is a barcode reader which reads the universal product code from each 
item (col.9, ln.32-36; col. 10, ln.21-31 and Fig.11, element #80). 

Referring to claim 15, Ashby, III et al. further disclose the system, wherein the 
storage mechanism is operative to store the spoken utterance corresponding to each 
item using the universal product code read by the barcode reader for each item as an 
index (memory, col.9, ln.31-47 and col.11. In. 1-3). 

Referring to claim 16, Ashby, III et al. further disclose the system, wherein the 
output device is operative to play back to the user the spoken utterance that is found 
during a search by the storage mechanism using a universal product code as an index 
when the universal product code is subsequently read from an item by the barcode 
reader(col.9, ln.31-34; col. 10, ln.66-col.11, ln.7). 

Referring to claim 21 , Ashby, III et al. disclose a apparatus for using speech for 
marking and subsequently identifying one or more items having barcodes respectively 
marked thereon, the apparatus comprising: 

a barcode reader (col.9, ln.32-36; col. 10, ln.21-31 andfig.11, element #80), the 
barcode reader being operative to input at least a portion of a barcode marked on an 
item (bar code, col .2, In. 1-5; col.5, ln.31-33 and col.9, ln.31-36); 
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a speech capturing device (nnicrophone, col.6, ln.38 and fig.1 1 , element #14), the 
speech capturing device being operative to input a spoken utterance from a user that 
corresponds to an item (col.1 , ln.63 - col .2, ln.1 and col. 5, ln.33-35); 

at least one processor (central computer, fig.1 1 , element #78), the processor 
being operatively coupled to the barcode reader and the speech capturing device and 
operative to: (i) associate in a database the barcode read from the item with the spoken 
utterance input from the user; and (ii) search the database for the spoken utterance 
when the barcode associated with the spoken utterance is subsequently read by the 
barcode reader; and coupled to the processor and operative to output the spoken 
utterance found during the search (col. 10, ln.66- col.1 1, ln.7). 

Referring to claim 25, Ashby et al. disclose an article of manufacture for using 
speech for marking and subsequently identifying one or more items having 
electronically-readable identifiers respectively marked thereon, comprising a machine 
readable medium containing one or more programs which when executed implement 
the steps of: 

inputting at least a portion of the electronically-readable identifier marked on an 
item (bar code, col.2. In. 1-5; col.5, ln.31-33 and col.9, ln.31-36); 

inputting from a user a spoken utterance that corresponds to the item (col.1, 
ln.63 -col.2, ln.1 and col.5, ln.33-35); 

associating the electronically-readable identifier input from the item with the 
spoken utterance input from the user (col.3, ln.36-39; coL5, ln.1 8-34 and col.9, ln.60- 
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65); and 

outputting the spoken utterance when the electronically-readable identifier 
associated with the spoken utterance is subsequently inputted (col. 3, ln.36-52 and col.9, 
ln.31-47). 



3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 6-11, 17-20 and 22-24 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Ashby, III et al. in view of Jong (U.S. Patent No. 6,173,250). 

Referring to claim 6, Ashby, III et al. suggest displaying textual associated with 
item on a local display (col. 10, ln.21-41). But, Ashby, III et al. do not specifically disclose 
converting the spoken utterances input by the user to produce their text. 

However, Jong teaches converting the spoken utterance to text (col.3, In. 36-43). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the apparatus of Ashby, III et al. to obtain 
text information by converting the user spoken utterances input, as taught by Jong, in 
order to produce text for deaf or hearing-impaired. 



Claim Rejections - 35 USC § 103 



Referring to claim 7, Ashby, III et al. do not specifically disclose converting the 
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spoken utterance to text by a speech recognition system. 

However, Jong teaches converting the spoken (utterance) to text by speech 
recognition system (speech recognition device, col. 3, ln.36-43). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the method of Ashby, III et al. to obtain text 
by a speech recognition system, as taught by Jong, in order to produce text for deaf or 
hearing-impaired. 

Referring to claim 8, Ashby, III et al. do not specifically disclose converting 
text obtained from Input speech back to speech but play back the original speech when 
the electronically-readable identifier associated with the spoken utterance is 
subsequently inputted. 

However, Jong teaches converting the user's original Input speech to text, storing 
text data in the memory (col.4. In. 14, 16 and fig.2, element #207), and then converting 
the text data into synthesized speech for output by speech output device (col.5, ln.11- 
30). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the Invention was made to modify the method of Ashby, III et al. to have the 
step of converting the user's input speech to text and text back to speech, as taught by 
Jong, when the electronically-readable identifier associated with the spoken utterance Is 
subsequently inputted, in order to provide multi-modal communication, not only for the 
deaf or hearing-impaired but also for the sight-impaired. 
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Referring to claim 9, Ashby, III et al. do not specifically teach converting the text 
back to speech by a text-to-speech system. 

However, Jong teaches converting the text obtained from input speech back to 
speech by a text-to-speech system (text to speech conversion device, col.5, ln.24-27 
and fig.2, element #206). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the method of Ashby, III et al. to have the 
text to speech system provide the speech information, as taught by Jong, in order to 
provide multi-modal communication not only for the deaf or hearing-impaired but also 
for the sight-impaired. 

Referring to claim 10, Ashby, III et al. do not specifically disclose speech-to-text 
conversion performed on a computing device remotely located with respect to a 
computing system performing the other steps. 

However, Jong suggest a method, wherein the speech-to-text conversion is 
performed on a computing device (speech recognition device, fig.2, element #203) 
remotely located with respect to a computing system (STT system, col .2, ln.55-56) 
performing the othersteps (col.2, ln.65; col.5. In. 13-33, fig.1 shows systems 100 and 
110. System 100 performs speech-to-text conversion and system 110 performs text-to- 
speech conversion. The two systems are remotely located, and they are connected by a 
transmitting network). 
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Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the method of Ashby, III et al. to have 
speech-to-text conversion performed on a computing device remotely located with 
respect to a computing system performing the other steps, as taught by Jong, in order 
to provide an efficient centralized way to transmit speech data and to facilitate update 
and maintenance. 

Referring to claim 1 1 , Ashby, III et al. disclose the computer device archives 
electronically-readable identifiers and associated spoken utterance (col. 10, ln.66 - 

col.11, ln.7). But, Ashby et al. do not specifically disclose a computing device that 
performs the speech-to-text conversion. 

However, Jong teaches the computing device that performs the speech-to-text 
conversion, and operation way of the speech-to-text conversion when it receives 
speech input signals (fig.3). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the method of Ashby, III et al. to have the 
computing device that performs the speech-to-text conversion, as taught by Jong, in 
order to carried and restore data in another system. 

Referring to claim 17, Ashby, III et al. teach text associated with the 
electronically-readable identifier, but do not specifically disclose converting the spoken 
utterances input by the user to text. 

However, Jong teaches converting the spoken utterance to text by a speech 
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recognition system (speech recognition device, col. 3, ln.36-43). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the system of Ashby, III et al. to obtain text 
by a speech recognition system, as taught by Jong, in order to produce the text for deaf 
or hearing-impaired. 

Referring to claim 18, Ashby, III et al. do not specifically disclose a text-to-speech 
system operatively coupled between the storage mechanism and the output device and 
operative to convert the text back to speech when the electronically-readable identifier 
associated with the spoken utterance is subsequently re-inputted and then outputting 
the converted speech. 

However, Jong teaches to convert the user's input speech to text and stored text 
data in the memory (col.4, In. 14, 16 and fig.2, element #207), the text data can be 
display or fonA/arded to the text to speech conversion device where the text data is 
converted into synthesized speech then output by speech output device (col.5, ln.1 1- 
30). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the system of Ashby, III et al. to have the 
step of converting the user's input speech associated with the electronically-readable 
identifier to text and text back to speech, as taught by Jong, in order to provide multi- 
modal communication not only for the deaf or hearing-impaired but also for the sight- 
impaired. 
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Referring to claim 19, Ashby, III et al. do not specifically disclose the system, 
comprising a speech recognition system, remotely located with respect to the system, 
for converting the spoken utterances input by the user to text. 

However, Jong suggest a method, wherein the speech-to-text conversion is 
performed on a computing device (speech recognition device, fig.2, element #203) 
remotely located with respect to a computing system (STT system, col.2. In. 55-56) 
performing the other steps (col.2, ln.65; col.5. In. 13-33, fig.1 shows systems 100 and 
110. System 100 performs speech-to-text conversion and system 110 performs text-to- 
speech conversion. The two systems are remotely located, and they are connected by a 
transmitting network). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the Invention was made to modify the method of Ashby, III et al. to have 
speech-to-text conversion performed on a computing device remotely located with 
respect to a computing system performing the other steps, as taught by Jong, in order 
to provide an efficient centralized way to transmit speech data and to facilitate update 
and maintenance. 

Referring to claim 20, Ashby, III et al. further disclose a computing device for 
remotely archiving the electronically-readable identifier/spoken utterance association 
(fig.11, elements #74 and #84, col. 10, ln.66-col.11, ln.7 and col.3, ln.38-43, the vocal 
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message can be recorded into a storage device corresponding to tlie bar-code label. 
The storage device is remote from the bar-code label (col. 10, In. 16-21)). 

Referring to claim 22, Ashby, III et al. suggest displaying text associated with 
item on a local display (col. 10, ln.21-41). But, Ashby, III et al. do not specifically disclose 
converting the spoken utterances input by the user to produce their text. 

However, Jong teaches converting the spoken (utterance) to text (col.3, ln.36- 

43). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the apparatus of Ashby, III et al. to obtain 
text information by converting the user spoken utterances Input, as taught by Jong, in 
order to produce text for deaf or hearing-impaired. 

Referring to claim 23, Ashby, III et al. disclose the apparatus, wherein the 
processor is operative to output the spoken utterance when the electronically-readable 
identifier associated with the spoken utterance Is subsequently Inputted (col. 9, ln.31- 
47). But, Ashby, III et al. do not specifically disclose a apparatus, wherein the processor 
Is operative to convert the text back to speech when the barcode associated with the 
spoken utterance Is subsequently read such that the converted speech is output by the 
speech outputting device. 

However, Jong teaches a processor operative to convert the user's Input speech 
to text and stored text data in the memory (col.4. In. 14, 16 and fig.2, element #207), the 
text data can be display or forwarded to the text to speech conversion device where the 
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text data is converted into synthesized speech then output by speech output device 
(col.5, In. 11-30). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the apparatus of Ashby, III et al. to have the 
processor is operative to convert the user's input speech to text and text back to 
speech, as taught by Jong, when the electronically-readable identifier associated with 
the spoken utterance is subsequently inputted in order to increase the convenience for 
the deaf or hearing-impaired by presenting text on the display. 

Referring to claim 24, Ashby, III et al. further disclose apparatus configured to be 
user-portable (fig.1 1 , top element #74). 

Conclusion 

5. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Dietz (U.S. Patent No. 6,175,820) teach a method for providing voice dynamics of 
human utterances converted to and represented by text with a data processing system. 

6. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to the examiner Vincent V. Iran whose E-mail address: 

Vincent.tran@USPT0.GOV . 
Phone number: (703) 305-1817 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mr. Talivaldis Ivars Smits, can be reached on (703) 306-301 1 . Any inquiry of 
a general natural or relating to the status of this application call receptionist at (703) 305 
-3900. 

7. Any response to this action should be mailed to: 

Commissioner of Patents and Trademarks 

P.O. Box 1450 

Alexandria, VA 22313-1450 
Or faxed to: 

(703) 872-9314 

Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal Dr, 
Arlington VA, Sixth Floor (Receptionist, Tel. No. 703-305-4700). 
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